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ABSTRACT 

Principals have two reasons to wonder about the whole 
concept of leadership. First, they are responsible for identifying leadership 
in others; and second, they must be analytical and reflective about their own 
capabilities. Consequently, there is always demand for valid and reliable 
measurement of leadership qualities. This issue reviews recent publications 
that address issues involved in measuring and evaluating leadership. The 
piiblications are: (1) ’'Measurements of I irsonality and Leadership: S:?me 
Relationships” (Frederick C. Wendel, Allan H. Schmidt, and James Loch); (2) 
"Assessing Leadership and Measuring Its Effects" (Richard L. Hughes, Robert 
C. Ginnett, and Gordon J. Curphy) ; (4) "Performance Appraisal and Selection 
of School Leaders: Selection Processes and Measurement Issues" (Kenneth 
Leithwood, Paul T. Begley, and J. Bradley Cousins) ; and (5) "Using 
Multi -Raters in Superintendent Evaluation" (Richard P. Santeusanio) . (LMI) 
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A fter yet another day of cri- 
sis management or battles 
with the bureaucracy, 
many principals must 
wonder about the whole concept of 
leadership. What they do to get 
through each day seems far 
removed from the visions of dynam- 
ic, inspirational leadership they 
once nurtured. In their darkest 
hours, they may wonder whether 
they really cue leaders. 

Prlnclp^s have two reasons to be 
troubled by these doubts. First, 
tliey are responsible for identifying 
leadership in others, especially 
when interviewing candidates for 
administrative or faculty positions. 
Given today’s quickly changing 
school environment, districts must 
identify candidates who can act 
with initiative, raise questions, and 
take intelligent risks. 

Second, as leaders themselves, 
principals must be analytical and 
reflective about their own capabill- 

Lari}' Lashway is a research analysf and 
writer hr the ERIC Cleannghouse on 
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ties. Are they leading their schools 
effectively? What skills need 
Improvement? How should they 
direct their personal and profes- 
sional growth? 

Because of such questions, there 
is always a strong market for 
instruments that promise valid and 
reliable measurement of leadership 
qualities. Leadership tests are 
abundant, but do they deliver on 
their claims? The fairest answer to 
that question is *Tes, but....” 
Measurements of leadership can 
provide valuable information not 
available from informal Interviews, 
observations, or intuition. However, 
users must select assessment pro- 
cedures wisely, recognizing that no 
single approach is perfect and that 
their choices must be tailored to 
the needs of the school. 

llie literature on these Issues is 
surprisingly limited, but the works 
discussed in this issue provide a 
variety of helpful perspectives. 

Frederick C. Wervdel and col- 
leagues review some basic concepts 
of measurement and apply them to 
leadership testing. 

Richard L Hughes and colleagues 
discuss some important techniques 



for measuring leadership qualities 
and the effects of leadership, 

Kenneth Leithwood and col- 
leagues examine the ways that 
schools measure leadership and 
provide recommendations for 
improvement. 

Richard P. Santeusanio describes 
the 360-degree feedback ^stem 
used in his school district and pro- 
vides guidelines for using multi- 
rater feedback. 

Frank Freeman and colleagues 
provide capsule descriptions of 
some commonly used leadership 
tests, and the ERfC Clearinghouse 
for Assessment and Evaluation 
offers an online database of leader- 
ship tests. 

Wendel, Frederick C.; Schmidt, 
Allan H.; and Loch, James. 
Meosuremenfs of Personality and 
Leadership: Some Relationships. 
Lincoln, Neb., 1992. 121 pages. 
ED 350 694. Available from: ERIC 
Document Reproduction Service, 
7420 Fullerton Rd., Suite 1 10, 
Springfield, VA 221 53-2852. 
800-443-3742. $20.40 plus 
postage. 

Measuring leadership is an 
attractive concept that seems to 
promise precise, scientifically valid 
“proof that a person has (or does 



BEST COPY AVAILABLE 



9 




- 1 










not have) appropriate qualities. 
Unfortunately, this promise can be 
misleading, since all measurement 
instruments have important limita- 
tions. In this monograph. Frederick 
Wendel and colleagues present a 
comprehensive but highly readable 
review of measurement concepts. 

The authors begin by cautioning 
that abstract concepts such as per- 
sonality and leadership are always 
more than the sum of precisely 
measured parts. We can, for exam- 
ple, accurately measure character- 
istics such as weight, height, and 
hair color, but the concept of beau- 
ty still remains in the eye of the 
beholder, 

A crucial point to understand is 
that every leadership test is based 
on some definition of leadership, 
and current tests are based on a 
wide range of definitions. Some 
focus on personal traits, such as 
drive or interpersonal skill. Some 
zero in on specific behaviors 
demonstrated by effective leaders. 
Others call attention to the way 
that leaders think or use power, 
and still others assume that leader- 
ship Varies with situations. 

Ihe practical implication is that 
principals must have a clear under- 
standing of how leadership is 
defined in their Institution. For 
example, if a school has adopted 
site-based decision making, an 
appropriate test will provide feed- 
back on how a loader facilitates the 
work of others. 

The authors provide capsule 
descriptions of 16 commonly used 
leadership tests, including informa- 
tion on their validity and reliability. 
Even more helpful Is a discussion 
of assessment centers, in which 
participants perform simulations of 
actual leadership tasks, thus pro- 
ducing “samples” rather than 
“signs” of leadership. (A paper -and- 
pencil test may reveal an Individ- 
ual’s knowledge of leadership 
principles but provide no Informa- 
tion on his or her ability to carry 
. out leadership tasks In a real-v/orld 
setting.) 

Wendel and colleagues cite stud- 



ies showing that 90 percent of 
those scoring at the highest level in 
the NASSP assessment centers 
receive similar high ratings In later 
job performance. Although assess- 
ment centers are considerably more 
expensive than paper-and-pencil 
tests, they may be cost-effective, 
especially for entry-level candidates 
with no track record. 

The authors concede that mea- 
suring leadership is still an inexact 
science requiring careful reflection 
and caution, yet the right test, “one 
that fits the purpose of the examin- 
er, if administered properly, if 
scored with exactitude, and if inter- 
preted correctly, will yield good 
results." Conversely, those who rely 
on intuition or unsystematic proce- 
dures are taking major risks. 



Hughes, Richard L.; Ginnett, Robert 
C.; and Curphy, Gordon J. 
"Assessing Leadership and 
Meosuring Its Effects." In 
Leadership: Enhancing fhe Lessons 
of Experience, Homewood, ill.: 

Irwin, 1993. 543 pages. ED 363 
927. Available /rom: Richard D. 
Irwin, Inc., 1 333 Burridge Parlcway, 
Burridge, IL 60521 . 630-789-4000. 
$42.95, 



In this chapter of a leadership 
textbook, the authors summarize 
the various ways that leadership 
qualities can be assessed and also 
discuss ways of measuring leader- 
ship effects. 

The critical-incidents technique 
asks subordinates, peers, or supe- 
riors to describe an incident in 
which the leader did a particularly 
good or bad job. This method typi- 
cally generates context- rich 
descriptions of the situation, the 
leader’s behaviors, and the impact 
of the leader’s actions on others. 
However, this technique requires 
considerable time, effort, and 
expense. In addition, Incidents may 
differ dramatically, making it diffi- 
cult to compare leaders by the 
same standard. The critical- inci- 



dents approach may be better suit- 
ed for Individual professional devel- 
opment rather than for selecting 
leaders. 

The interview Is a more familiar 
tool for assessing leadership skill. 
Structured interviews, in which 
leaders are asked a predetermined 
set of questions, are generally 
preferable to informal, unstruc- 
tured interviews, since a common 
set of questions makes compar- 
isons much easier. 

Observation is another common 
technique that can be structured 
{with a predetermined set of cate- 
gories) or unstructured (with 
observers simply recording any- 
thing that happens). Obser\^ations 
can yield excellent information 
about how leaders actually behave, 
but systematically observing and 
recording on-the-job behavior is 
both difficult and time-consuming. 

Two other types of assessment 
are the familiar paper-and-pencil 
measures (such as surveys and 
questionnaires) and the assess- 
ment center, which challenges 
examinees with a series of inter- 
views and realistic leadership 
tasks, using trained assessors to 
rate their performance. 

While these methods will gener- 
ate information on the qualities 
and skills that leaders possess, it 
is also possible to assess the 
effects of leadership. Here, the key 
issue Is identifying the criteria by 
which performance is to be judged; 
that is. what will we accept as evi- 
dence that the leader is success- 
ful? One common answer is that 
successful leaders are those whose 
unit performs well. (An obrious 
example, in regard to a school 
leader, would be test scores). 

The problem with such criteria Is 
that they may be affected by factors 
other than the leader’s skills. For 
example, test scores may be influ- 
enced far more by student back- 
ground than by the principal's 
leadership skills. The authors rec- 
ommend the use of muluple crite- 
ria, with emphasis on those most 
responsive to leader actions. 
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Leithwood, Kenneth; Begley, Paul t; 
and Cousins, J. Bradley. 
^Performance Appraisol and 
Selection of Schm Leaders: 

Selection Processes ond 
Measurement Issues/ In Developing 
Expert leadership far Future 
Schools. London: Fdmer Press, 

1994. 331 pages. ED 394 172. 
Available from: Palmer Press, Tovlor 
& Francis, Inc., 1 900 Frost Roaa, 
Suite 101, Bristol, PA 19007. 215- 
785-5800. $27 paperback. 

This selection reviews the ways 
that schools measure leadership 
and provides guidelines for improv- 
ing selection, appraisal, and profes- 
sional development. 

Lelthwood and his colleagues 
note that the literature on leader 
selection is surprisingly thin, con- 
sidering the consequences of mak- 
ing a poor choice. Many selection 
processes are unsystematic, relying 
more on subjective perceptions 
than hard evidence. The interview 
appears to be the most commonly 
used tool, with standardized instru- 
ments Infrequently used. The pre- 
dictive power of such methods is 
questionable. 

However, after examining 14 for- 
mal Instruments designed to mea- 
sure school leadership, the autliors 
identified a number of potential 
problems. First, some of the 
instruments based their questions 
on observation of typical rather 
than expert leaders. The fact that 
many leaders behave in a certain 
way does not make that behavior 
desirable. 

Second, the quantitative dimen- 
sions of tests may obscure the 
many opportunities for subjectivity 
and human judgment in their 
development and administration. 
No test is sufficiently valid to be 
the sole measure for important 
decisions. 

Third, even when tests are reli- 
able. testers may not be. Tests can 
be used inappropriately, adminis- 



tered carelessly, or the results sim- 
ply ignored. 

Lelthwood and colleagues present 
several guidelines to improve the 
use of leadership assessment in 
schools: 

• The first priority is careful 
preparation. Schools should specify 
their testing policies and proce- 
dures, stating clearly why they are 
testing and what they hope to 
learn. While results may sometimes 
be used to help make personnel 
decisions, the major goal of mea- 
‘?urlng leadership should be profes- 
sional growth and Improvement. 

• Those responsible for selecting 
or evaluating leaders should be weU 
trained in assessment procedures. 

• Schools can improve credibility 
of their assessment process by 
using multiple data sources rather 
than any single technique. Formal 
instruments may be most useful as 
an early screening tool. 

• Leadership assessment should 
not be left to single Individuals. 
Panels involving people with differ- 
ent roles will ensure fairness as 
well as the perception of fairness. 
Leaders should be Involved with 
their assessment, and should 
rece/ive feedback that allows them 
to develop plans for future growth. 

The authors acknowledge that 
their recommendations are ambi- 
tious, and may best be viewed as 
worthwhile directions for the future 
rather than as reforms to be 
achieved overnight. But they argue 
that the stakes are too high to 
ignore these issues. 

Santeusanio, Richard P. "Using 
Multi-Raters in Superintendent 
Evaluation/ The School 
Adm/n/s/rafor 54:3 (March 1997): 

1 2. Available from: American 
Association of School Administrators 
Distribution Center, PO. Box 41 1 , 
Annapolis Jet., MD 20701 -041 1 . 
888-782-2272. 



One of the newer approaches to 
leadership assessment is *360- 
degree feedback," which relies on 
ratings from multiple sources. 
Typically, a leader’s performance 
and characteristics will be 
assessed by subordinates, peers, 
and superiors. 

Superintendent Richard 
Santeusanio succinctly describes 
the five-step process used in his 
Danvers, Massachusetts, school 
district for administrators and 
teachers. The first step is to identi- 
fy the competencies related to the 
Job and format them into a survey. 
The second step is to select the 
evaluation team. For Santeusanio’s 
own assessment, the team consist- 
ed of 21 people, including board 
members, principals, cvuriculum 
specialists, teachers, and parents. 
Teams with somewhat different 
makeup might be used to assess 
other administrators. 

The third and fourth steps are to 
conduct and score the survey. The 
final— and most important— step is 
to develop an action plan based on 
the strengths and weaknesses iden- 
tified by the survey. Santeusanio 
reports that he and others in the 
district have found this collective 
feedback to be more valid and 
meaningful than other methods. It 
has led not only to a clearer under- 
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standing of performance standards, 
but has fostered change for profes- 
sional improvement 
He notes that 360-degree feed- 
back solves some problems com- 
mon with single-source 
evaluations, including lack of fair- 
ness, accuracy, credibility, and 
usefulness to the person being 
evaluated. His district has found 
it to be both balanced and time- 
efficient. 



Freeman, Frank H.; Knott, Katherine 
B.; and Schwartz, Mary K. 
Leadership Education: A Source 
Book, 1996-1997. Sixth Edition, 
Volume 2. Greensboro, N.C,: Center 
for Creative Leadership, 1 996. 352 
pages. Available from: Center for 
Creative Leadership, One 



Leadership Place, RO. Box 26300, 
Greensboro, NC 27438-6300. 910- 
288-7210. $35.00 plus $4.00 ship- 
ping and handling. [Also available: 
Volume 1, 457 pages, $35.00; both 
volumes, $60.00.1 

Finding an appropriate leader- 
ship instrument is a potentially 
time-consuming task, traditionally 
requiring a hit-or-miss search 
through many sources. In recent 
years, however, a number of groups 
and publications have begun mak- 
ing this information more easily 
accessible. 

The Center for Creative 
Leadership’s Source Book includes 
a section with brief descriptions of 
73 commonly used leadership tests, 
including target audiences, admin- 
istration procedures, and com- 



ments on their validity and reliabil- 
ity. Vendor addresses are provided. 

Another source is the test-locator 
website maintained by the BUC 
Clearinghouse for Assessment ond 
Evaluation (http://erlcae.net). The 
database provides quick access to 
information on more than 10,000 
Instruments of various types. For 
each test there is a brief descriptive 
abstract and the publisher’s 
address. In addition, the site can 
point the searcher to published 
reviews of some tests, and it also 
has a section providing guidelines 
for choosing instruments. 

Later this year, the ERIC 
Clearinghouse on Educational 
Management will publish a book on 
measuring leadership that will be 
aimed at meeting the specific needs 
of school leaders. IZ 
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